TITLE OF THE INVENTION 
IMAGE PROCESSING APPARATUS AND METHOD 



FIELD OF THE INVENTION 

The present invention relates to a field of image 
processing and, more particularly, to Image encoding 
and decoding. 

BACKGROUND OF THE INVENTION 

Along with recent remarkable progress in 
computers and networks, various kinds of information 
such as character data, image data, and audio data are 
stored and transmitted in computers or between networks. 
Of these data, an image and especially a multilevel 
image contains a very large quantity of information. 
In storing/transmitting the image, the enormous amount 
of data poses a problem 

Hence, for an application purpose such as an 
image database which allows image browsing through a 
network, images in the device for storing them are 
often compression-coded. To browse these images, all 
the compression-coded data must be transmitted and 
decoded. If the band of the network is insufficient, 
it takes a long time to display the image. 

In such a case, only part of the 
compression-coded data is transmitted to display the 
image at a resolution lower than the original 



resolution. For, e.g., JPEG that is a 
compression-coding scheme recommended by the ISO and 
ITU-T, a hierarchical encoding scheme is defined 
whereby an image can be decoded at a resolution lower 
5 than the original resolution by a factor of a power of 
2. 

In JPEG, however, the encoding processing is 
complex, a reduced image or thumbnail image must be 
generated and encoded, and, for each layer of 

10 resolution necessary for decoding, the difference 

between a result obtained by decoding the reduced image 
and an image obtained by reducing the original image to 
that resolution must be encoded. 

An encoding scheme that has recently received a 

15 great deal of attention is a scheme using discrete 

wavelet transformation. Fig. 17A is a block diagram 
showing the basic arrangement of a 

compression-coding/decoding apparatus based on discrete 
wavelet transformation. Referring to Fig. 17A, an 

20 image input section 1 outputs an image signal to be 

compression-coded, and transformation is performed by a 
discrete wavelet transform section 2 on the output side. 
The discrete wavelet transform section 2 executes 
2-dimensional discrete wavelet transformation for the 

25 image signal and outputs transform coefficients. The 
transform coefficients are put together in units of 
predetermined frequency bands (to be referred to as 
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subfoands hereinafter), and quantized and converted into 
quantization indices by a quantization section 3 on the 
output side. The quantization indices are encoded by 
an entropy encoding section 4, so a code sequence is 
5 output. 

Fig. 17D is a block diagram showing the 
arrangement of a decoding apparatus for decoding the 
thus generated code sequence. Referring to Fig. 17D, 
quantization indices decoded by an entropy decoding 
10 section 5 are reconstructed to transform coefficients 

by an inverse quantization section 6 on the output side. 
The reconstructed transform coefficients are inversely 
transformed by an inverse discrete wavelet transform 
section 7 , so the image signal is reconstructed and 
15 output from an image output section 8. 

Fig. 17B is a view showing the structure of 
subbands generated by the discrete wavelet transform 
section 2. Referring to Fig. 17B, a subband LL is the 
subband with the lowest frequency and can also be 
20 regarded as an image signal whose resolution is lowered 
by passing the original image through a low-pass filter 
and subsampling the image signal. Hence, when only the 
subband LL is decoded without decoding all subbands in 
decoding the image signal, and the decoded signal is 
25 normalized into the dynamic range of the original image 
signal, an image reduced to 1/4 the original image in 
both the vertical and horizontal directions can be 



- 3 - 



obtained . 

When an image with a resolution higher by one 
level is necessary, subbands lower by one level, i.e., 
subbands HL2, LH2, and HH2 are decoded and inversely 
5 transformed together with the subband LL, as shown in 
Fig. 17C. An image whose resolution is lowered to 1/2 
in both the vertical and horizontal directions is 
obtained, as shown in Fig. 17C. 

As described above, in the encoding scheme using 
0 discrete wavelet transformation, a reduced image can be 
generated from a code sequence using the subband 
structure by transformation. 

However, as described above, in the method using 
the subband structure of discrete wavelet 
5 transformation, the quality of the generated reduced 
image is not always sufficiently high. Especially, 
when the image contains information such as characters 
or graphic patterns, such patterns cannot be 
discriminated in some cases because an image without 
) any high-frequency component is obtained by the above 
method . 



SUMMARY OF THE INVENTION 
It is a main object of the present invention to 
provide an image processing technique capable of 
maintaining the quality of an entire image or a 
necessary portion in encoding/decoding the image in 
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providing a reduced image . 

According to the present invention, there is 
provided an image processing apparatus comprising: 

encoding means for executing predetermined 
5 sequence transformation for an input image and 

bit-plane-encoding an obtained transform coefficient; 

storing means for storing encoded data obtained 
by the encoding means; 

recognition means for recognizing a first image 
10 quality of the image to be displayed by a predetermined 
display device; 

output means for reading out, from the storing 
means, data necessary to reconstruct an image having a 
predetermined image quality in the encoded data and 
15 outputting the data; 

decoding means for decoding the output encoded 
data; and 

adjustment means for adjusting the image decoded 
by the decoding means to make an image quality of the 
20 image match the first image quality, 

wherein the recognition means instructs the 
output means to read out encoded data corresponding to 
a decoded image having a second image quality in the 
held encoded data on the basis of a predetermine 
25 condition, and 

the adjustment means adjusts the image obtained 
by decoding, by the decoding means, the encoded data 
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output by the output means, to make an image quality of 
the image to be displayed by the display device match 
the first image quality. 

According to the present invention, there is also 
provided an image processing method comprising: 

the encoding step of executing predetermined 
sequence transformation for an input image and 
bit-plane-encoding an obtained transform coefficient; 

the step of storing encoded data obtained in the 
encoding step in storage means; 

the recognition step of recognizing a first image 
quality of the image to be displayed by a predetermined 
display device; 

the output step of reading out, from the storage 
means, data necessary to reconstruct an image having a 
predetermined image quality in the encoded data and 
outputting the data; 

the decoding step of decoding the output encoded 
data; and 

the adjustment step of adjusting the image 
decoded in the decoding step to make an image quality 
of the image match the first image quality, 

wherein the output step comprises the step of 
reading out encoded data corresponding to a decoded 
image having a second image quality in the held encoded 
data on the basis of a predetermine condition, and 

the adjustment step comprises the step of 



adjusting the image obtained by decoding, in the 
decoding step, the encoded data output in the output 
step, to make an image quality of the image to be 
displayed by the display device match the first image 
quality. 

According to the present invention, there is also 

provided a program wherein 

the program causes a computer to function as 
encoding means for executing predetermined 

sequence transformation for an input image and 

bit-plane-encoding an obtained transform coefficient; 

storing means for storing encoded data obtained 

by the encoding means ; 

recognition means for recognizing a first image 

quality of the image to be displayed by a predetermined 

display device- 
output means for reading out, from the storing 

means, data necessary to reconstruct an image having a 

predetermined image quality in the encoded data and 

outputting the data; 

decoding means for decoding the output encoded 
data; and 

adjustment means for adjusting the image decoded 
by the decoding means to make an image quality of the 
image match the first image quality, 

wherein the recognition means instructs the 
output means to read out encoded data corresponding to 



a decoded image having a second image quality in the 
held encoded data on the basis of a predetermine 
condition, and 

the adjustment means adjusts the image obtained 
5 by decoding, by the decoding means, the encoded data 

output by the output means, to make an image quality of 
the image to be displayed by the display device match 
the first image quality. 

According to the present invention, there is also 
10 provided an image processing apparatus comprising an 
encoding section and a decoding section, 

the encoding section comprising 

means for encoding an input image to generate 
encoded data, 

15 means for receiving a designation of an image 

quality for display of the input image, and 

means for outputting, of the encoded data, 
encoded data necessary to display the input image at an 
image quality equal to or higher than the designated 
20 image quality, and 

the decoding section comprising 

means for decoding the encoded data output from 
the encoding section to generate image data, and 

means for, when an image based on the image data 
25 has an image quality higher than the designated image 
quality, converting the image data into image data 
having the designated image quality. 
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According to the present invention, there is also 
provided an image processing apparatus comprising: 

means for encoding an image to generate encoded 

data; 

5 means for receiving a designation of an image 

quality for display of the image; and 

means for outputting, of the encoded data, 
encoded data necessary to display the image at an image 
quality equal to or higher than the designated image 
10 quality. 

According to the present invention, there is also 
provided an image processing apparatus for decoding 
encoded data output from an encoding device for 
encoding an image, comprising: 
15 means for designating an image quality of the 

image for the encoding device; 

means for decoding the encoded data output from 
the encoding device to generate image data; and 

means for, when an image based on the image data 
20 has an image quality higher than the designated image 
quality, converting the image data into image data 
having the designated image quality. 

According to the present invention, there is also 
provided an image processing method comprising the 
25 encoding step and the decoding step, 

the encoding step comprising the steps of 
encoding an input image to generate encoded data, 
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receiving a designation of an image quality for 
display of the input image, and 

outputting, of the encoded data, encoded data 
necessary to display the input image at an image 
quality equal to or higher than the designated image 
quality, and 

the decoding step comprising the steps of 

decoding the encoded data output from an encoding 
section to generate image data, and 

when an image based on the image data has an 
image quality higher than the designated image quality, 
converting the image data into image data having the 
designated image quality. 

According to the present invention, there is also 
provided an image processing method comprising the 
steps of: 

encoding an image to generate encoded data; 

receiving a designation of an image quality for 
display of the image; and 

outputting, of the encoded data, encoded data 
necessary to display the image at an image quality 
equal to or higher than the designated image quality. 

According to the present invention, there is also 
provided an image processing method of decoding encoded 
data output from an encoding device for encoding an 
image, comprising the steps of: 

designating an image quality of the image for the 



encoding device; 

decoding the encoded data output from the 
encoding device to generate image data; and 

when an image based on the image data has an 
5 image quality higher than the designated image quality, 
converting the image data into image data having the 
designated image quality. 

According to the present invention, there is also 
provided a program wherein 
10 the program causes a computer to function as an 

encoding section and a decoding section, 

the encoding section comprising 

means for encoding an input image to generate 
encoded data, 

15 means for receiving a designation of an image 

quality for display of the input image, and 

means for outputting, of the encoded data, 
encoded data necessary to display the input image at an 
image quality equal to or higher than the designated 
20 image quality, and 

the decoding section comprising 

means for decoding the encoded data output from 
the encoding section to generate image data, and 

means for, when an image based on the image data 
25 has an image quality higher than the designated image 
quality, converting the image data into image data 
having the designated image quality. 
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According to the present invention, there is also 
provided a program which causes a computer to function 
as : 

means for encoding an image to generate encoded 

data; 

means for receiving a designation of an image 
quality for display of the image; and 

means for outputting, of the encoded data, 
encoded data necessary to display the image at an image 
quality equal to or higher than the designated image 
quality. 

According to the present invention, there is also 
provided a program which, to decode encoded data output 
from an encoding device for encoding an image, causes a 
computer to function as: 

means for designating an image quality of the 
image for the encoding device; 

means for decoding the encoded data output from 
the encoding device to generate image data; and 

means for, when an image based on the image data 
has an image quality higher than the designated image 
quality, converting the image data into image data 
having the designated image quality. 

According to the present invention, there is also 
provided an image processing apparatus for encoding an 
image and outputting encoded data, comprising: 

determination means for determining a type of the 



image; 

encoding means for executing encoding including 
discrete wavelet transformation for the image to 
generate the encoded data of the image; 
5 means for receiving an input of a resolution of a 

decoded image, which is to be used upon decoding the 
encoded data; and 

output means for outputting, of the generated 
encoded data, data necessary to generate the de^cod^d 
10 image having the input resolution, 

wherein for a specific type of image, the output 
means outputs, of the encoded data, data necessary to 
generate the decoded image having a resolution higher 
than the input resolution. 
15 According to the present invention, there is also 

provided an image processing apparatus for encoding an 
image and outputting encoded data, comprising: 

means for segmenting the image into a plurality 
of regions of a predetermined unit; 
20 determination means for determining a type of 

each of the segmented images; 

encoding means for executing encoding including 
discrete wavelet transformation for each of the 
segmented images to generate the encoded data of each 
25 of the images; 

means for receiving an input of a resolution of a 
decoded image, which is to be used upon decoding the 
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encoded data; and 

output means for outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 
5 wherein for the region corresponding to a 

specific type of image, the output means outputs, of 
the encoded data, data necessary to generate the 
decoded image having a resolution higher than the input 
resolution. In addition, there is provided an image 
10 processing apparatus for decoding the encoded data 
output by this image processing apparatus, wherein 

for the region corresponding to the specific type 
of image, the decoded image is formed at a resolution 
higher than that of the remaining regions. 
15 According to the present invention, there is also 

provided an image processing apparatus for encoding an 
image and outputting encoded data, comprising: 

first encoding means for segmenting the image 
into a plurality of subbands by discrete wavelet 
20 transformation; 

means for segmenting an image of each subband 
into regions of a predetermined unit; 

determination means for determining a type of the 
image of each region; 
25 second encoding means for executing quantization 

and entropy encoding for the image of each region to 
generate encoded data of each region; 
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means for receiving an input of a resolution of a 
decoded image, which is to be used upon decoding the 
encoded data; and 

output means for outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein for a specific type of image, the output 
means outputs, of the encoded data, data necessary to 
generate the decoded image having a resolution higher 
than the input resolution. In addition, there is 
provided an image processing apparatus for decoding the 
encoded data output by this image processing apparatus, 
wherein 

for the region corresponding to the specific type 
of image, the decoded image is formed at a resolution 
higher than that of the remaining regions. 

According to the present invention, there is also 
provided an image processing apparatus for encoding an 
image and outputting encoded data, comprising: 

means for segmenting the image into a plurality 
of regions of a predetermined unit; 

determination means for determining a type of 
each of the segmented images; 

encoding means for executing encoding including 
discrete wavelet transformation for each of the 
segmented images to generate the encoded data of each 
of the images; 



means for receiving an input of a resolution of a 
decoded image, which is to be used upon decoding the 
encoded data; and 

output means for outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein for the region corresponding to a 
specific type of image, the output means additionally 
outputs, of the encoded data, data necessary to 
generate the decoded image having a resolution higher 
than the input resolution. 

According to the present invention, there is also 
provided an image processing method of encoding an 
image and outputting encoded data, comprising: 

the determination step of determining a type of 
the image; 

the encoding step of executing encoding including 
discrete wavelet transformation for the image to 
generate the encoded data of the image; 

the step of receiving an input of a resolution of 
a decoded image, which is to be used upon decoding the 
encoded data; and 

the output step of outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein the output step comprises the step of, 
for a specific type of image, outputting, of the 
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encoded data, data necessary to generate the decoded 
image having a resolution higher than the input 
resolution . 

According to the present invention, there is also 
5 provided an image processing method of encoding an 
image and outputting encoded data, comprising: 

the step of segmenting the image into a plurality 
of regions of a predetermined unit; 

the determination step of determining a type of 
10 each of the segmented images; 

the encoding step of executing encoding including 
discrete wavelet transformation for each of the 
segmented images to generate the encoded data of each 
of the images; 

15 the step of receiving an input of a resolution of 

a decoded image, which is to be used upon decoding the 
encoded data; and 

the output step of outputting, of the generated 
encoded data, data necessary to generate the decoded 

20 image having the input resolution, 

wherein the output step comprises the step of, 
for the region corresponding to a specific type of 
image, outputting, of the encoded data, data necessary 
to generate the decoded image having a resolution 

25 higher than the input resolution. In addition, there 
is provided an image processing method of decoding the 
encoded data output by this image processing method, 
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wherein 

for the region corresponding to the specific type 
of image, the decoded image is formed at a resolution 
higher than that of the remaining regions. 
5 According to the present invention, there is also 

provided an image processing method of encoding an 
image and outputting encoded data, comprising: 

the first encoding step of segmenting the image 
into a plurality of subbands by discrete wavelet 
10 transformation; 

the step of segmenting an image of each subband 
into regions of a predetermined unit; 

the determination step of determining a type of 
the image of each region; 
15 the second encoding step of executing 

quantization and entropy encoding for the image of each 
region to generate encoded data of each region; 

the step of receiving an input of a resolution of 
a decoded image, which is to be used upon decoding the 
20 encoded data; and 

the output step of outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein the output step comprises, for a specific 
25 type of image, outputting, of the encoded data, data 
necessary to generate the decoded image having a 
resolution higher than the input resolution. In 
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addition, there is provided an image processing method 
of decoding the encoded data output by this image 
processing method, wherein 

for the region corresponding to the specific type 
5 of image, the decoded image is formed at a resolution 
higher than that of the remaining regions. 

According to the present invention, there is also 
provided an image processing method of encoding an 
;?f image and outputting encoded data, comprising: 

j ^ 10 the step of segmenting the image into a plurality 

*l: of regions of a predetermined unit; 

the determination step of determining a type of 

O each of the segmented images ; 

UI 

111 the encoding step of executing encoding including 

r] 15 discrete wavelet transformation for each of the 

segmented images to generate the encoded data of each 
of the images; 

the step of receiving an input of a resolution of 
a decoded image, which is to be used upon decoding the 
20 encoded data; and 

the output step of outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein the output step comprises the step of, 
25 for the region corresponding to a specific type of 

image, additionally outputting, of the encoded data, 
data necessary to generate the decoded image having a 
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resolution higher than the input resolution. 

According to the present invention, there is also 
provided a program wherein 

to encode an image to output encoded data, the 
5 program causes a computer to function as: 

determination means for determining a type of the 

image ; 

encoding means for executing encoding including 
discrete wavelet transformation for the image to 
10 generate the encoded data of the image; 

means for receiving an input of a resolution of a 
decoded image, which is to be used upon decoding the 
encoded data; and 

output means for outputting, of the generated 
15 encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein for a specific type of image, the output 
means outputs, of the encoded data, data necessary to 
generate the decoded image having a resolution higher 
20 than the input resolution. 

According to the present invention, there is also 
provided a program wherein 

to encode an image to output encoded data, the 
program causes a computer to function as: 
25 means for segmenting the image into a plurality 

of regions of a predetermined unit; 

determination means for determining a type of 
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each of the segmented images; 

encoding means for executing encoding including 
discrete wavelet transformation for each of the 
segmented images to generate the encoded data of each 
5 of the images; 

means for receiving an input of a resolution of a 
decoded image, which is to be used upon decoding the 
encoded data; and 

output means for outputting, of the generated 
10 encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein for the region corresponding to a 
specific type of image, the output means outputs, of 
the encoded data, data necessary to generate the 
15 decoded image having a resolution higher than the input 
resolution. In addition, there is provided a program 
wherein, 

to decode encoded data output by causing a 
computer to execute this program, the program causes 
20 the computer to function as: 

means for, for the region corresponding to the 
specific type of image, forming the decoded image at a 
resolution higher than that of the remaining regions. 

According to the present invention, there is also 
25 provided a program wherein 

to encode an image to output encoded data, the 
program causes a computer to function as: 
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first encoding means for segmenting the image 
into a plurality of subbands by discrete wavelet 
transformation; 

means for segmenting an image of each subband 
into regions of a predetermined unit; 

determination means for determining a type of the 
image of each region; 

second encoding means for executing quantization 
and entropy encoding for the image of each region to 
generate encoded data of each region; 

means for receiving an input of a resolution of a 
decoded image, which is to be used upon decoding the 
encoded data; and 

output means for outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein for a specific type of image, the output 
means outputs, of the encoded data, data necessary to 
generate the decoded image having a resolution higher 
than the input resolution. In addition, there is 
provided a program wherein 

to decode encoded data output by causing a 
computer to execute this program, the program causes 
the computer to function as: 

means for, for the region corresponding to the 
specific type of image, forming the decoded image at a 
resolution higher than that of the remaining regions. 



According to the present invention, there is also 
provided a program wherein 

to encode an image to output encoded data, the 
program causes a computer to function as: 
5 means for segmenting the image into a plurality 

of regions of a predetermined unit; 

determination means for determining a type of 
each of the segmented images; 

encoding means for executing encoding including 
10 discrete wavelet transformation for each of the 

segmented images to generate the encoded data of each 
of the images; 

means for receiving an input of a resolution of a 
decoded image / which is to be used upon decoding the 
15 encoded data; and 

output means for outputting, of the generated 
encoded data, data necessary to generate the decoded 
image having the input resolution, 

wherein for the region corresponding to a 
20 specific type of image, the output means additionally 
outputs, of the encoded data, data necessary to 
generate the decoded image having a resolution higher 
than the input resolution. 

Other features and advantages of the present 
25 invention will be apparent from the following 

description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
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the same or similar parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings, which are incorporated 
5 in and constitute a part of the specification, 

illustrate embodiments of the invention and, together 
with the description, serve to explain the principles 
of the invention. 

Figs. 1A to 1C are block diagrams of an image 
10 processing apparatus according to the first embodiment 
of the present invention; 

Fig. 2A is a view showing a subband segmented 
into code blocks CB; 

Figs. 2B and 2C are explanatory views of 
15 operation of bit plane encoding by an entropy encoding 
section 4; 

Figs. 3A to 3E are schematic views of a code 
sequence generated and output by the image processing 
apparatus shown in Fig. 1; 
20 Figs. 4A and 4B are views showing a display 

resolution and output resolution of encoded data output 
from a transmission section 14; 

Fig. 4C is a view showing an output code sequence 
when an image is segmented into a plurality of tiles in 
25 the third embodiment; 

Figs. 5A to 5E are views showing the structure of 
a code sequence by a code sequence construction section 
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11 according to the second embodiment of the present 
inventions- 
Fig. 6A is a view showing the structure of a code 
sequence by the code sequence construction section 11 
according to the second embodiment of the present 
invention; 

Fig. 6B is a view showing a display resolution 
and applied level in inverse discrete wavelet 
transformation in the second embodiment of the present 
invention ; 

Fig. 7 is a block diagram of an image processing 
apparatus according to the fourth embodiment of the 
present invention; 

Fig. 8 is an explanatory view showing a form 
wherein an image is segmented into tiles; 

Figs. 9A to 9C are explanatory views of a 
discrete wavelet transform section 2; 

Figs. 10A and 10B are explanatory views of 
processing by an entropy encoding section 4; 

Figs. 11A to 11D are explanatory views of code 
sequences generated by the image processing apparatus 
according to the fourth embodiment of the present 
invention; 

Figs. 12A and 12B are explanatory views of a code 
sequence generated by a code sequence construction 
section 11; 

Figs. 13A and 13B are explanatory views of 



processing by an entropy decoding section 5; 

Figs. 14A and 14B are explanatory views of an 
inverse discrete wavelet transform section 7; 

Figs. 15A and 15B are explanatory views of block 
5 segmentation of a subband and its code sequence in the 
fifth embodiment of the present invention; 

Figs. 16A to 16C are explanatory views of a 
decoded image composition procedure in the fifth 
embodiment of the present invention; and 
10 Figs. 17A to 17D are explanatory views of an 

image processing apparatus of a prior art. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Preferred Embodiments of the present invention 
15 will now be described in detail in accordance with the 
accompanying drawings . 
<First Embodiment > 

Figs. 1A to 1C are block diagrams of an image 
processing apparatus according to an embodiment of the 
20 present invention. An outline of operation of the 

image processing apparatus will be described first with 
reference to Figs. 1A to 1C. 

Referring to Figs. 1A to 1C, an image input from 
an image input section 1 is compression-coded by an 
25 image encoder ENC. The generated code sequence is 

output to a code sequence construction section 11 on 
the output side. The code sequence construction 
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section 11 constructs the received code sequence by a 
predetermined method and outputs it to a code sequence 
storage section 10 or transmission section 14. The 
code sequence construction section 11 also has a 
5 function of changing the structure of a code sequence 
stored in the code sequence storage section 10 and 
storing or transmitting the code sequence. 

The code sequence storage section 10 is a storage 
medium such as a memory or hard disk for storing a 

10 compression-coded image. The code sequence stored in 
the code sequence storage section 10 is transmitted to 
the decoding side through the transmission section 14, 
reconstructed into an image by an image decoder DEC, 
and displayed or output from an image output section 12 

15 through an image construction section 8. 

An image quality setting section 13 sets the 
quality of the image to be output from the image output 
section 12 on the basis of a user input or a 
predetermined input (not shown) . In this embodiment, 

20 the image quality is defined as the resolution of an 
image displayed. Hence, information related to the 
resolution designated by the image quality setting 
section 13 is independently sent to the code sequence 
construction section 11, and the code sequence 

25 construction section 11 outputs a predetermined code 

sequence to the transmission section 14 on the basis of 
this information. Operation of each section of this 



- 27 - 



image processing apparatus will be described below in 
detail . 

The image input section 1 receives an image 
having a predetermined resolution from an input device 
5 (not shown) and outputs the image to the encoder ENC. 
The input image can be either a color image or a 
monochrome grayscale image. The encoding processing to 
be described later assumes that the input image is a 
monochrome grayscale image. However, even when the 
10 input image is a color image, the present invention can 
be applied by independently performing a series of 
processing operations for each color component of the 
color image. 

The input image is subjected to discrete wavelet 
15 transformation by a discrete wavelet transform section 
2. The discrete wavelet transform section 2 executes 
two-dimensional discrete wavelet transform processing 
for the received image signal and calculates and 
outputs transform coefficients. Fig. 9A shows the 
20 basic arrangement of the discrete wavelet transform 
section 2. The received image signal is stored in a 
memory 201, sequentially read out and transformed by a 
processing section 202, and written in the memory 201 
again . 

25 Fig. 9B shows the arrangement of the processing 

section 202. The input image signal is separated into 
even and odd address signals by a combination of a 
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delay element and down sampler, and filter processing 
is executed by two filters p and u. Referring to 
Fig. 9B, s and d denote low- and high-pass coefficients 
in decomposing a one-dimensional image signal to one 
level and are given by 

d(n) - x(2*n + 1) - floor ( (x(2*n) 
+ x(2*n + 2) ) /2) 

. - - (1) 

s(n) - x(2*n)_ + floor ((d(n - 1) + d(n))/4) 

... (2) 

where x(n) is the image signal to be transformed. 

With the above processing, one-dimensional 
discrete wavelet transformation for the image signal is 
performed. Two-dimensional discrete wavelet 
transformation is executed by sequentially performing 
one-dimensional transformation for the image in the 
horizontal and vertical directions. Details of this 
processing are known, and a description thereof will be 
omitted here. Fig. 9C shows the structure of transform 
coefficient groups of two levels obtained by the 
two-dimensional transform processing. The image signal 
is decomposed into subbands HH1, HL1, LH1, . . . , LL. The 
coefficient of each subband is output to a quantization 
section 3 on the output side. 

The quantization section 3 quantizes the received 
coefficients by a predetermined quantization step and 
outputs indices corresponding to the quantized values. 



Quantization is described by 

q = sign ( c) floor (abs (c) /A) ...(3) 
sign (c) = 1 ; c >= 0 ... (4) 

sign(c) = -1 ; c < 0 ...(5) 
5 where c is the coefficient to be quantized. In this 
embodiment , 1 is included as a value A, In this case, 
the transform coefficients input to the quantization 
section 3 are directly output to an entropy encoding 
section 4 on the output side without performing actual 
10 quantization. 

The entropy encoding section 4 decomposes the 
received quantization indices into bit planes in units 
of rectangular regions (to be referred to as code 
blocks hereinafter) obtained by segmenting the subband 
15 to a predetermined size, sequentially performs binary 
arithmetic encoding for the bit planes, and outputs a 
code stream. Fig. 2A is a view showing a subband 
segmented into code blocks CB. Referring to Fig. 2A, 
for example, the subband LL is segmented into four code 
20 blocks CBO to CB3, and the quantization indices in the 
code blocks are subjected to bit plane encoding in the 
order of the code blocks CBO, CB1, CB2, and CB3. 

The entropy encoding section 4 scans the entire 
subband first to obtain a maximum value M and 
25 calculates the number S of bits necessary to express 
the maximum quantization index M by 

S = ceil (log2 (abs (M) ) ) -.-(6) 
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where ceil (x) is the smallest integer value of integers 
equal to or larger than x. In addition, even in each 
code block, the maximum number SB of bits is calculated 
from the maximum quantization index value. Fig. 2B is 
5 a view for explaining the bit plane encoding operation 
by the entropy encoding section 4. In this example, 
three non-zero quantization indices are present in a 
code block region having a size of 4 X 4, and the 
indices have values "+13", n -6", and "+3", respectively. 

10 The entropy encoding section 4 scans the entire code 

block to obtain a maximum value MB and then calculates 
the number SB of bits necessary to express the maximum 
quantization index by 

SB = ceil (log2 (abs (MB) ) ) ...(7) 

15 Referring to Fig. 2B, since the maximum 

coefficient value is 13, the number SB of bits is 4. 
The 16 quantization indices in the code block are 
processed in units of four bit planes, as shown in 
Fig. 2C. First, the entropy encoding section 4 

20 performs binary arithmetic encoding for each bit of the 
most significant bit plane (MSB in Fig. 2C) and outputs 
a bit stream. The level of bit plane is lowered by one, 
each bit in the bit planes is encoded until the bit 
plane to be processed reaches a least significant bit 

25 plane LSB, and a bit stream is output to the code 

sequence construction section 11. At this time, for 
the sign of each quantization index, immediately after 
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the first non-zero bit is detected by bit plane 
scanning, the sign of the quantization index is 
entropy-encoded. The number of actually encoded bit 
planes is also output to the code sequence construction 
5 section 11 and contained in a code sequence as a 
parameter, as will be described later. In this 
embodiment, each bit plane is executed by one path. 
;JB5 However, each bit plane may be segmented into a 

plurality of encoding paths and executed. In this case, 
*ll 10 a corresponding encoding path is treated as one unit. 

The code sequence construction section 11 
w constructs a code sequence of the received codes having 

O predetermined header information and the like and 

111 outputs the code sequence to the code sequence storage 

Q 15 section 10 or transmission section 14. 

Figs. 3A to 3E are schematic views showing the 
structures of the thus generated and output code 
sequence* Fig. 3A shows the entire structure of the 
code sequence. Reference symbol MH denotes a main 
20 header, TH; a tile header; and BS, a bit stream. As 

shown in Fig. 3B, the main header MH contains the size 
of the image to be encoded (the number of horizontal 
and vertical pixels) , the size when the image is 
segmented into tiles as a plurality of rectangular 
25 regions, the number of components, which represents the 
number of color components, and component information 
representing the size and bit accuracy of each 
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component. In this embodiment , since the image is not 
broken up into tiles,, the tile size and image size take 
the same value. When the image to be processed is a 
monochrome grayscale image / the number of components is 
5 1. 

Fig. 3C shows the structure of the tile header TH. 
The tile header TH is formed from a tile length 
including the code length and header length of the tile, 
and an encoding parameter for the tile. The encoding 

10 parameter includes the level of discrete wavelet 

transformation and the filter type. Fig. 3D shows the 
structure of encoded data in this embodiment. 
Referring to Fig. 3D, the encoded data are put together 
in units of subbands which are arranged in ascending 

15 order of resolution sequentially from a subband with a 
low resolution. In each subband, the encoded data are 
arranged from the most significant bit plane to the 
least significant bit plane. 

Encoded data related to the bit plane or encoding 

20 path of each code block are classified into a plurality 
of layers. Each layer is formed from encoded data 
related to an arbitrary number of bit planes or 
encoding paths from each code block. For example, when 
one layer contains encoded data related to one bit 

25 plane from each code block, one layer corresponds to 
one bit plane. Since the layers are formed starting 
from the upper bit plane toward the lower bit plane, an 
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upper layer always contains encoded data of a bit plane 
located at a place on the upper side of that contained 
in the lower layer. 

The codes of each layer are put together in units 
5 of code blocks, as shown in Fig. 3E. A field PH 

contains the difference between a corresponding bit 
plane S and the maximum bit plane SB of each code block, 
information for designating a code block contained in 
the layer, and the like. Referring to Fig. 3E f a layer 

10 L-l contains codes of four code blocks. A code block 
whose corresponding code is not present in this layer 
can be specified from code block designation in the 
field PH. The code sequence storage section 10 stores 
the thus generated code sequence. 

15 The function of the code sequence construction 

section 11 and a method of displaying an image 
according to the final encoded data output from the 
code sequence construction section 11 in a reduced 
state will be described next. 

20 Referring to Fig. 1A, the display resolution of 

the decoded image is designated from an input device 
(not shown) to the image quality setting section 13. 
As the input device, in addition to a keyboard, mouse, 
or switch provided in the image processing apparatus, a 

25 computer connected to the image processing apparatus 
through a network can be used. 

The display resolution corresponds to one of 
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reduced images that can be composited and generated 
from the subband structure obtained by discrete wavelet 
transformation as shown in Fig. 9C. The image quality 
setting section 13 determines the output resolution for 
5 the code sequence to be output from the code sequence 
construction section 11 on the basis of the input 
display resolution and gives an output instruction to 
the code sequence construction section 11. The output 
instruction may be given through a system (not shown) 

10 or the transmission section 14, as shown in Fig. 1A. 

On the basis of the instruction from the image 
quality setting section 13, the code sequence 
construction section 11 reads out encoded data with the 
output resolution from the code sequence previously 

15 stored in the code sequence storage section 10 and 

outputs the encoded data to the transmission section 14. 
Figs. 4A and 4B are views showing the display 
resolution and output resolution at this time. 
Referring to Figs. 4A and 4B, the display resolution 

20 corresponds to the resolution of the subband LL shown 
in Fig. 4A. However, since the output resolution 
determined by the image quality setting section 13 is a 
resolution higher by one level, the output code 
sequence contains the encoded data of the subbands HL2, 

25 LH2, and HH2 in addition to that of the subband LL, as 
shown in Fig. 4B. 

The output code sequence is decoded by the 
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decoder DEC through the transmission section 14 to 
reconstruct the image signal. Operation of the decoder 
DEC and a series of operation until image display will 
be described below. 
5 An entropy decoding section 5 extracts necessary 

information by analyzing the header of the received 
code sequence r and decodes and outputs the bit stream 
of the received code sequence. Decoding is 
sequentially performed from the upper bit plane to the 

10 lower bit plane by analyzing the code sequence shown in 
Fig. 3E and extracting the encoded data in a 
predetermined code block from each layer. 

The decoding procedure is shown in Figs. 13A and 
13B. Fig. 13A is a view showing a flow in which code 

15 blocks in the subband to be decoded are sequentially 
decoded in units of bit planes, and the quantization 
indices are finally reconstructed. The bit planes are 
decoded in the direction indicated by the arrow in 
Fig. 13A. The restored quantization indices are output 

20 to an inverse quantization section 6. 

The inverse quantization section 6 reconstructs 
the discrete wavelet transform coefficients from the 
received quantization index on the basis of 

c' = A*q ; q * 0 ... (8) 

25 c r - 0 ; q = 0 ... (9) 

where q is the quantization index, A is the 
quantization step which has the same value as that used 
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for encoding, and c' is the reconstructed transform 
coefficient, i.e., the reconstructed coefficient s or d 
in encoding. The transform coefficient c r is output to 
an inverse discrete wavelet transform section 7 on the 
output side. 

Figs. 14A and 14B are views showing the 
arrangement and processing of the inverse discrete 
wavelet transform section 7. Referring to Fig. 14A, 
the received transform coefficient is stored in a 
memory 701. A processing section 702 executes 
one-dimensional inverse discrete wavelet transformation 
and performs the processing while sequentially reading 
out the transform coefficients from the memory 701, 
thereby executing two-dimensional inverse discrete 
wavelet transformation. The two-dimensional discrete 
wavelet transformation is executed by a procedure 
opposite to the forward procedure. Details are known, 
and a description thereof will be omitted. Fig. 14B 
shows the processing block of the processing section 
702. The received transform coefficients undergo 
filter processing by two filters u and p and are 
superposed after upsampling to output an image signal 
x T . These processing operations are performed by 

x T (2*n) = s'(n) - floor((d , (n - 1) + d ( (n))/4) 

. . . (10) 

x' (2*n + 1) - d 1 (n) + f loor ( (x 1 (2*n) 
+ x< (2*n + 2) ) /2) 



The image signal reclaimed in this way is output 
to the image construction section 8 on the output side. 
All subbands contained in the received code sequence 
5 are decoded by the above-described series of processing 
operations, and the image signal having the output 
resolution previously determined by the image quality 
setting section 13 is reconstructed by inverse discrete 
wavelet transformation and output to the image 

10 construction section 8. 

The image construction section 8 receives the 
display resolution previously input from the image 
quality setting section 13, compares the display 
resolution with the resolution input from the decoder 

15 DEC, and when the resolution of the received image 

signal equals the display resolution, directly outputs 
the image signal to the image output section 12. 

On the other hand, when the resolution of the 
image received from the decoder DEC is different from 

20 the display resolution, the image construction section 
8 applies a predetermined resolution transformation 
method to the received image to generate an image 
corresponding to the display resolution and outputs the 
image to the image output section 12. 

25 As described above, in this embodiment, the image 

quality setting section 13 instructs the code sequence 
construction section 11 to output encoded data related 
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to a resolution higher than the display resolution by 
one level. The image construction section 8 converts 
the resolution of the image decoded by the decoder DEC 
such that the resolution matches the display resolution, 
5 and outputs the image. 

When an image having a resolution higher than the 
display resolution is decoded, an image signal 
containing a high-frequency component necessary to 
express an accurate image can be obtained. As compared 

10 to a case wherein an image is decoded from only a 

subband with the display resolution, the image quality 
of a character portion or edge portion contained in the 
image can be greatly improved. 

In this embodiment, encoded data related to a 

15 resolution higher than the display resolution by one 
level is output. However, encoded data related to a 
resolution higher by two or more levels may be output 
in accordance with the required image quality. 

In the above-described embodiment, the decoder 

20 DEC may not decode all bit planes, as needed. More 
specifically, for only the subband LL in the output 
code sequence shown in Fig. 4A, all layers contained in 
the code sequence are decoded, though for the subbands 
HL2, LH2, and HH2, decoding may be stopped at a 

25 predetermined layer, and subsequent processing may be 
performed . 

With this processing, the image quality of a 
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reduced image is improved by reconstructing a 
high-frequency component at a predetermined accuracy, 
while the time required for entropy decoding can be 
shortened. At this time, the image quality setting 
5 section 13 may instruct the decoder DEC to output only 
some upper layers for a predetermined high-frequency 
component. In this case, since all layers need not be 
decoded by the decoder DEC, the processing can be 
executed at a higher speed. 

10 <Second Embodiment> 

In the above-described first embodiment, the 
output code sequence is arranged in units of subbands. 
However, another arrangement may be employed. A case 
wherein a code sequence constructed by a code sequence 

15 construction section 11 has another form will be 
described below. 

Figs. 5A to 5E are views showing the structure of 
a code sequence by the code sequence construction 
section 11 according to this embodiment. The 

20 structures shown In Figs. 5A to 5C are the same as in 
the first embodiment, and a detailed description 
thereof will be omitted. Referring to Fig. 5D, encoded 
data are put together in units of layers, and each 
layer contains a predetermined amount of encoded data 

25 in each subband. 

The encoded data in each subband is formed from 
encoded data corresponding to a predetermined number of 



- 40 - 



bit planes or encoding paths from the code block in the 
subband. Operation of each section in outputting an 
image when such a code sequence is formed will be 
described below. 
5 An image quality setting section 13 outputs an 

instruction to the code sequence construction section 
11 to output a code sequence related to a predetermined 
number of layers. Fig. 6A shows the structure of the 
code sequence at this time, in which the code sequence 

10 of three upper layers is output. On the other hand, as 
in the first embodiment, the image quality setting 
section 13 receives and stores a display resolution 
from an input device (not shown) , and also outputs the 
number of levels of inverse discrete wavelet 

15 transformation to an inverse discrete wavelet transform 
section 7 of a decoder DEC. 

The decoder DEC decodes the code sequence 
according to the same procedure as described above, so 
quantization indices are reconstructed. Additionally, 

20 in this embodiment, for a series of received transform 
coefficients, the inverse discrete wavelet transform 
section performs inverse transformation of the inverse 
transform level previously input from the image quality 
setting section 13 and outputs the generated image 

25 signal to an image construction section 8. The range 
of the image signal obtained by inverse transformation 
is adjusted to the range of the original signal. 
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For example, assume that when the data of subband 
decoded from the three upper layers corresponds to 
2-level discrete wavelet transformation, and the 
display resolution corresponds to the resolution of a 
5 subband LL, as shown in Fig. 6B, the number of levels 
of the inverse discrete wavelet transformation 
designated by the image quality setting section 13 is 1, 
an image signal with a resolution higher by one level 
than that of the subband LL is reconstructed by the 

10 inverse transformation, as shown in Fig, 6B. 

The reconstructed image signal has a resolution 
twice the display resolution. The signal obtained by 
1-level inverse transformation is discrete wavelet 
transform coefficients and can be treated as an image 

15 signal by normalizing the range of the signal in 
accordance with the range of the output device. 

The image signal obtained by the above method is 
output to an image output section 12 after its 
resolution is converted such that it matches the 

20 display resolution by an image construction section 8, 
as in the first embodiment. 
<Third Embodiment> 

In the above-described first and second 
embodiments, the image is not segmented into tiles. 

25 However, the present invention is not limited to this, 

and an image may be segmented into a plurality of tiles. 
In addition, the output resolution set by the image 
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quality setting section may be changed in units of 
tiles . 

In the third embodiment, an image quality setting 
section 13 outputs both a display resolution and an 
5 output resolution to a code sequence construction 

section 11. The code sequence construction section 11 
stores an image type of each tile in advance by a 
predetermined method and reconstructs the code sequence 
such that a code sequence with the output resolution is 
10 output for a predetermined tile, and a code sequence 

with the display resolution is output for the remaining 
tiles . 

Fig. 4C is a view showing an output code sequence 
when an image is segmented into a plurality of tiles. 
15 Referring to Fig. 4C, in tiles 0 and 1, only encoded 
data of a subband LL is contained in the output code 
sequence. However, for tile 5, encoded data of a 
subband for a resolution higher by one level is also 
contained . 

20 To display an image from the code sequence with 

such a structure, an image construction section 8 
compares the display resolution with the decoding 
resolution of each tile. If the display resolution is 
different from the resolution of the decoded image, the 

25 resolution is converted. Otherwise, the image is 

directly output so that an entirely reduced image can 
be displayed on an image output section 12. 
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< Fourth Embodiment > 

Fig. 7 is a block diagram of an image processing 
apparatus according to the fourth embodiment of the 
present invention. First, an outline of operation of 
this image processing apparatus will be described with 
reference to Fig. 7. An example of application field 
of this image processing apparatus is a copying machine 
connected to a network or a digital camera. 

Referring to Fig. 7, an image input from an image 
input section 1 is compression-coded by an image 
encoder ENC, and the generated code sequence is output 
to a code sequence construction section 11 on the 
output side. The code sequence construction section 11 
constructs the received code sequence by a 
predetermined method and outputs the code sequence to a 
code sequence storage section 10 or an external device. 

The code sequence storage section 10 is a storage 
medium such as a memory or hard disk for storing a 
compression-coded image. The code sequence stored in 
the code sequence storage section 10 is read out as 
needed, and the code sequence transmitted to the 
decoding side is reconstructed into an image by an 
image decoder DEC, and displayed or output from an 
image output section 12. 

Assume that the image processing apparatus is a 
copying machine connected to a network. The image 
input section 1 corresponds to a scanner portion 



prepared in the copying machine. The image decoder DEC 
is, e.g., incorporated in a computer connected through 
the network. The image output section 12 corresponds 
to the monitor of the computer. 
5 In this case, image data encoded in the copying 

machine is stored in the copying machine. A code 
sequence is sent to the computer through the network on 
the basis of an instruction from the computer on the 
network such that a reduced image of the stored image 

10 data can be displayed on the computer. When a large 
amount of documents is to be processed by the copying 
machine, the progress of processing can be monitored 
from the computer connected through the network. 

Assume that the image processing apparatus is a 

15 digital camera. Both the image encoder ENC and the 
image decoder DEC are arranged in the digital camera 
main body. The image input section 1 corresponds to an 
image sensing device such as a CCD. The code sequence 
storage section 10 corresponds to a memory in the 

20 camera main body. The image output section 12 
corresponds to a compact liquid crystal monitor 
provided in the camera main body. 

In this case, a sensed image is compression-coded 
by the image encoder ENC, stored in the memory in the 

25 camera main body, and displayed on the liquid crystal 

monitor as needed. At this time, a reduced image within 
the size of the liquid crystal monitor is generated 
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from a code sequence and displayed. 

In either example, to display the 
compression-coded image, a reduced image having a 
resolution lower than the original resolution is 
displayed. Operation of each section of the image 
processing apparatus will be described below in detail. 

An image signal is segmented into predetermined 
regions, in this case, rectangular tiles each having a 
predetermined size and output by the image input 
section 1. Fig. 8 is a view showing an example in 
which an image is segmented into tiles. An image I to 
be processed is segmented into 16 tiles having the same 
size. The thus segmented image signal is sequentially 
output in units of tiles to a discrete wavelet 
transform section 2 of the image encoder ENC and a 
region determination section 9 on the output side. 

The region determination section 9 discriminates 
the characteristics of the image input in units of 
tiles. In this embodiment, the region determination 
section 9 determines types such as a natural image and 
text image as types of image. For this discrimination 
scheme, a known discrimination scheme can be used. For 
example, the histogram of pixel values in the image 
input in units of tiles is calculated, and its 
distribution is used. The discrimination result is 
output to the code sequence construction section 11. 
The code sequence construction section 11 will be 



described later. 

The image signal output from the image input 
section 1 in units of tiles is subjected to discrete 
wavelet transformation by the discrete wavelet 
5 transform section 2 on the output side. The discrete 
wavelet transform section 2 executes two-dimensional 
discrete wavelet transform processing for the received 
image signal and calculates and outputs transform 
coefficients. Fig. 9A shows the basic arrangement of 
10 the discrete wavelet transform section 2. The received 
image signal is stored in a memory 201, sequentially 
read out and transformed by a processing section 202 f 
and written in the memory 201 again. 

Fig. 9B shows the arrangement of the processing 
15 section 202. The input image signal is separated into 
even and odd address signals by a combination of a 
delay element and down sampler^ and filter processing 
is executed by two filters p and u. Referring to 
Fig. 9B, s and d denote low- and high-pass coefficients 
20 in decomposing a one-dimensional image signal to one 
level and are given by 

d(n) = x(2*n + 1) - f loor ( (x (2*n) 
+ x (2*n + 2) ) /2) 

. . . (12) 

25 s(n) = x(2*n) + floor ((d(n - 1) + d(n))/4) 

. . . (13) 

where x(n) is the image signal to be transformed. 
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With the above processing, one-dimensional 
discrete wavelet transformation for the image signal is 
performed. Two-dimensional discrete wavelet 
transformation is executed by sequentially performing 
5 one-dimensional transformation for the image in the 
horizontal and vertical directions. Details of this 
processing are known, and a description thereof will be 
omitted here. Fig. 9C shows the structure of transform 
coefficient groups of two levels obtained by the 
10 two-dimensional transform processing. The image signal 
is decomposed into subbands HH1, HL1, LH1,..., LL. The 
coefficient of each subband is output to a quantization 
section 3 on the output side. 

The quantization section 3 quantizes the received 
15 coefficients by a predetermined quantization step and 
outputs indices corresponding to the quantized values. 
Quantization is described by 

q = sign (c) floor (abs (c) /A) ...(14) 

sign(c) = 1 ; c >= 0 ...(15) 
20 sign(c) = -1 ; c < 0 ...(16) 

where c is the coefficient to be quantized. In this 
embodiment, 1 is included as a value A. In this case, 
the transform coefficients input to the quantization 
section 3 are directly output to an entropy encoding 
25 section 4 on the output side without performing actual 
quantization . 

The entropy encoding section 4 decomposes the 



- 48 - 



received quantization indices into bit planes, performs 
binary arithmetic encoding in units of bit planes, and 
outputs a code sequence. 

Figs. 10A and 10B are views for explaining 
5 operation of the entropy encoding section 4. In this 
example, three non-zero quantization indices are 
present in a subband region having a size of 4 x 4, and 
the indices have values "+13", "-6", and "+3", 
respectively. The entropy encoding section 4 scans 

10 this region to obtain a maximum value M and then 

calculates the number S of bits necessary to express 
the maximum quantization index by 

S - ceil (log2 (abs (M) ) ) ...(17) 
where ceil (x) is the smallest integer value of integers 

15 equal to or larger than x. Referring to Figs. 10A and 
10B, since the maximum coefficient value is 13, the 
number S of bits is 4. The 16 quantization indices in 
the sequence are processed in units of four bit planes, 
as shown in Fig. 10B. First, the entropy encoding 

20 section 4 performs binary arithmetic encoding for each 
bit of the most significant bit plane (MSB in Fig. 2C) 
and outputs a bit stream. The level of bit plane is 
lowered by one, each bit in the bit planes is encoded 
until the bit plane to be processed reaches a least 

25 significant bit plane (LSB in Fig. 10B) , and a code 

stream is output to the code output section 5. At this 
time, for the sign of each quantization index, 
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immediately after the first non-zero bit is detected by 
bit plane scanning, the sign of the quantization index 
is entropy-encoded. The thus generated code sequence 
is output to the code sequence construction section 11 
5 on the output side. 

The type of tile output from the region 
determination section 9 is input to the code sequence 
construction section 11. The code sequence 
.1* construction section 11 constructs the final code 

Y„{ 10 sequence on the basis of the data from the region 

determination section 9 and entropy encoding section 4. 
'*l The constructed code sequence is stored in a code 

;^ sequence storage section 10. 

Figs. 11A to 11D are schematic views showing the 
Ml 15 structures of the thus generated and output code 
M sequence. Fig. 11A shows the entire structure of the 

code sequence. Reference symbol MH denotes a main 
header, TH; a tile header; and BS, a bit stream. 

As shown in Fig. 11B, the main header MH contains 
20 the size of the image to be encoded (the number of 
horizontal and vertical pixels) , the size when the 
segmented image is further segmented into tiles as a 
plurality of rectangular regions, the number of 
components, which represents the number of color 
25 components, and component information representing the 
size and bit accuracy of each component. In this 
embodiment, since the image is not segmented into 
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(sub) tiles, the tile size and image size take the same 
value. When the image to be processed is a monochrome 
grayscale image, the number of components is 1. 

Fig. 11C shows the structure of the tile header 
5 TH. The tile header TH is formed from a tile length 
including the bit stream length and header length of 
the tile, and an encoding parameter for the tile. The 
encoding parameter includes the level of discrete 
wavelet transformation and the filter type. The tile 

10 type is data representing the tile type determined by 
the above-described region determination section 9. 

Fig. 11D shows the structure of the bit stream in 
this embodiment. Referring to Fig. 11D, the bit 
streams are put together in units of subbands which are 

15 arranged in ascending order of resolution sequentially 
from a subband with a low resolution. In each subband, 
the bit streams are arranged from the most significant 
bit plane to the least significant bit plane, and the 
codes are arranged in units of bit planes. 

20 The function of the code sequence construction 

section 11 and a method of displaying an image 
according to the final encoded data output from the 
code sequence construction section 11 in a reduced 
state will be described next. 

25 Referring to Fig. 7, the display resolution of 

the decoded image is designated from an input device 
(not shown) to the code sequence construction section 
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11. That is, the code sequence construction section 11 
also functions as a means for receiving an input of the 
resolution of the decoded image. As the input device, 
in addition to a keyboard, mouse, or switch provided in 
5 the image processing apparatus, a computer connected to 
the image processing apparatus through a network can be 
used. 

The resolution corresponds to one of reduced 
images that can be composited and generated from the 

10 subband structure obtained by discrete wavelet 

transformation as shown in Fig. 9C. The code sequence 
construction section 11 reconstructs and outputs a code 
sequence to be output to the decoding side on the basis 
of the code sequence stored in the code sequence 

15 storage section 10. 

Figs. 12A and 12B are views showing the 
relationship between the image segmented into tiles and 
the code sequence reconstructed by the code sequence 
construction section 11 in correspondence with the 

20 image. Fig. 12A shows the image segmented into tiles. 
Tiles 5 and 6 are discriminated as text regions by the 
region determination section 9. Fig. 12B shows part of 
the code sequence reconstructed for this image. When 
the resolution designated from the external device to 

25 the code sequence construction section 11 corresponds 

to a subband LL by discrete wavelet transformation, the 
bit stream of each tile in the reconstructed code 
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sequence is formed from only codes of the subband LL if 
the tile is a natural image. However, for the two 
tiles shown in Fig. 12A, not only codes of the subband 
LL but also codes corresponds to three subbands HL2, 
5 LH2, and HH are contained. 

To reconstruct a code sequence, the tile type in 
the tile header TH of the code sequence previously 
stored in the code sequence storage section 10 is 
iji referred to, and if the tile is a natural image, only 

j„> 10 the subband LL is output; otherwise, the subbands HL2, 
;j| LH2, and HH2 are output in addition to the subband LL. 

The resultant code sequence is sent to the decoding 
|U side through a predetermined transmission path and 

,^5 decoded by a decoder DEC. Decoding processing and 

15 image reconstruction of this embodiment will be further 
described below. 

An entropy decoding section 5 decodes the bit 
stream of the received code sequence in units of bit 
planes and outputs the bit stream. The decoding 
20 procedure is shown in Figs. 13A and 13B. Fig. 13A is a 
view showing a flow in which a region of the subband to 
be decoded is sequentially decoded in units of bit 
planes, and the quantization indices are finally 
reconstructed. The bit planes are decoded in the 
25 direction indicated by the arrow in Fig. 13A. The 
reconstructed quantization indices are output to an 
inverse quantization section 6. 
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The inverse quantization section 6 reconstructs 
the discrete wavelet transform coefficients from the 
received quantization index on the basis of 

c f - A*q ; q * 0 ... (18) 

c 1 = 0 ; q = 0 ... (19) 

where q is the quantization index, A is the 
quantization step which has the same value as that used 
for encoding, and c ? is the reconstructed transform 
coefficient, i.e., the reconstructed coefficient s or d 
in encoding. The transform coefficient c' is output to 
an inverse discrete wavelet transform section 7 on the 
output side. 

Figs. 14A and 14B are views showing the 
arrangement and processing of the inverse discrete 
wavelet transform section 7. Referring to Fig. 14A, 
the received transform coefficients are stored in a 
memory 701. A processing section 702 executes 
one-dimensional inverse discrete wavelet transformation 
and performs the processing while sequentially reading 
out the transform coefficients from the memory 701, 
thereby executing two-dimensional inverse discrete 
wavelet transformation. The two-dimensional discrete 
wavelet transformation is executed by a procedure 
opposite to the forward procedure. Details are known, 
and a description thereof will be omitted. Fig. 14B 
shows the processing block of the processing section 
702. The received transform coefficients undergo 



filter processing by two filters u and p and are 
superposed after upsampling to output an image signal 
x' . These processing operations are performed by 

x 1 (2*n) = s'(n) - floor((d , (n - 1) + d f (n))/4) 
5 , . • (20) 

x T (2*n + 1) - d T (n) + f loor ( (x f <2*n) 
+ x f (2*n + 2) ) /2) 

- . - (21) 

The image signal reconstructed in this way is 

10 output to an image construction section 8 on the output 
side. The above-described series of processing 
operations are independently performed in units of 
tiles of the code sequence, and all subbands contained 
in the received bit stream are decoded. In this 

15 embodiment, only the subband LL is used to decode a 

tile as a natural image, and the subbands LH2, HL2, and 
HH2 are used to decode a tile as a text image. Hence, 
in this embodiment, a tile corresponding to a text 
image is reconstructed as an image having a resolution 

20 twice that of a tile corresponding to a natural image 
in both the vertical and horizontal directions. 

The image construction section 8 constructs an 
image to be finally displayed on the image output 
section 12 on the basis of the type of each tile and 

25 the resolution required of the reconstructed image. 
When the tile is a natural image, the image 
construction section 8 directly outputs the 
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reconstructed image signal to the image output section 
12 on the output side without any processing. On the 
other hand, when the tile is a text image, the image 
construction section 8 performs resolution 
5 transformation such that the resolution of the tile 
becomes 1/2 in the horizontal and vertical directions 
and then outputs the image signal to the image output 
section 12. A known technique such as cubic 
interpolation can be used to convert the resolution, 

10 and a detailed description thereof will be omitted. 

As described above, in this embodiment, when an 
image is segmented into tiles, and an independently 
encoded code sequence is decoded to generate a reduced 
image, a code in a high-frequency region is added for a 

15 specific tile to generate an image in a wider band, and 
then, reduction processing is executed while 
maintaining prescribed image quality, thereby 
generating the image to be finally displayed. 

With this processing, the reduced image can be 

20 generated while maintaining specific information, 

unlike a case wherein the subband structure by normal 
discrete wavelet transformation is directly used. For 
this reason, a portion such as a text image that 
particularly requires detailed information can be 

25 displayed at a high image quality. In addition, since 
only necessary parts of the compression-coded data are 
contained in the code sequence, the transmission 
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overhead in transmitting the code sequence can be 
minimized, and the image can be quickly displayed. 
<Fifth Embodiment > 

In the above-described fourth embodiment, before 
5 encoding, the image is segmented into tiles, and 
processing is executed in units of tiles. The same 
effect as described above can be obtained even using a 
structure obtained by segmenting a subband by discrete 
wavelet transformation into blocks. An embodiment for 

10 such a structure will be described below. The 

arrangement of the apparatus of this embodiment is the 
same as in that shown in Fig. 7, and only different 
points will be described. 

Figs. 15A and 15B are views showing a block as a 

15 unit for encoding and a generated code sequence. On 
the left side of Fig, 15A, an image is segmented into 
seven subbands by 2-level discrete wavelet 
transformation. Each subband is further segmented into 
regions of a predetermined unit, i.e., blocks having 

20 the same size, which are indicated by broken lines on 
the right side of Fig. 15A. 

The blocks are independently encoded. The 
encoding method is the same as that described in the 
first embodiment. The blocks are independently 

25 quantized and encoded in units of bit planes. 

Fig. 15B is a view showing the structure of a bit 
stream in the code sequence obtained by encoding. The 
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bit stream is formed from encoded data that are 
independent in units of blocks. The codes of each 
block are arranged from the upper bit plane to the 
lower bit plane. The remaining portions are the same 
5 as in the first embodiment. 

In this embodiment , in reconstructing an image , a 
code sequence construction section 11 specifies a block 
corresponding to a text image and constructs a code 
sequence to be sent to the decoding side, as shown in 

10 Fig. 16A. More specifically, assume that a shaded 

portion is a text image, as shown on the left side of 
Fig. 16A. When 2-level discrete wavelet transformation 
is performed for that region, the region corresponds to 
a shaded portion on the right side of Fig. 16A. 

15 To display a reduced image having the resolution 

of a subband LL, the code sequence construction section 
11 adds not only all blocks of the subband LL but also 
blocks including a text region in subbands HL2, LH2, 
and HH2 to the code sequence to be output. 

20 HL2: bO, bl 

LH2: b2, b3 
HH2: b4, b5 

An image decoder DEC on the decoding side decodes 
the thus constructed code sequence to generate an image 
25 (to be referred to as an image A hereinafter) having 

the same resolution as that of the subband LL first, as 
shown in Fig. 16C. On the other hand, the decoder DEC 
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generates an image (to be referred to as an image B 
hereinafter) having a resolution twice that of the 
subband LL using the decoded subband LL and the 
subbands HL2 , LH2 , and HH2 . In the subbands except the 
5 subband LL, only two of the four blocks have 

coefficients. Inverse discrete wavelet transformation 
is performed by regarding all coefficients of the 
missed portions as 0. 

Q An image construction section 8 performs 

'II 

10 predetermined resolution conversion for the image B 
obtained by the above operation to generate an image 
ip (to be referred to as an image C hereinafter) having 

the same resolution as that of the subband LL, 

• ^ % 

jjj composites a portion D in the image C, which 

\h\ 15 corresponds to a text image, with the previously 

7[ generated image A to generate an image to be finally 

output, and outputs the image to an image output 

section 12. 

The fourth and fifth preferable embodiments of 
20 the present invention have been described above- In 
the fourth and fifth embodiments, for a portion 
corresponding to a text image, an image having a 
resolution higher than that of the remaining portions 
is generated and then reduced by resolution conversion, 
25 thereby generating an image. 

However, depending on contents, the text portion 
may contain important information, and for the 
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remaining portions, only outlines may suffice. In this 
case, the text portion may be composited with the 
remaining portions at a double resolution without being 
reduced . 

5 The preferred embodiments of the present 

invention have been described above. The present 
invention may be applied to a system constituted by a 
plurality of devices (e.g., a host computer, an 
interface device, a reader, a printer, and the like) or 

10 an apparatus comprising a single device (e.g., a copying 
machine, a facsimile apparatus, or the like) . 

The object of the present invention is achieved 
even by supplying software program codes for 
implementing the functions of the above-described 

15 embodiments to the system or apparatus through a 

storage medium (or recording medium) which stores the 
program codes and causing the computer (or a CPU or 
MPU) of the system or apparatus to execute the program 
codes* In this case, the program codes read out from 

20 the storage medium implement the functions of the 
above-described embodiments by themselves, and the 
program codes and a program product related to the 
program codes such as the storage medium which stores 
the program codes constitute the present invention. The 

2 5 functions of the above-described embodiments are 

implemented not only when the readout program codes are 
executed by the computer but also when the operating 
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system (OS) running on the computer performs part or all 
of actual processing on the basis of the instructions of 
the program codes. 

The functions of the above-described embodiments 
are also implemented when the program codes read out 
from the storage medium are written in the memory of a 
function expansion card inserted into the computer or a 
function expansion unit connected to the computer, and 
the CPU of the function expansion card or function 
expansion unit performs part or all of actual processing 
on the basis of the instructions of the program codes. 

As many apparently widely different embodiments of 
the present invention can be made without departing from 
the spirit and scope thereof, it is to be understood 
that the invention is not limited to the specific 
embodiments thereof except as defined in the appended 
claims . 



